Field of the Invention
The present inventions relate to at least one image processing apparatus, at least one image processing method, and at least one storage medium. In particular, the present inventions are suitable for use in correcting an image signal.
Description of the Related Art
As digitization of information, an image is handled as signal values, and there are therefore various processes for correcting a captured image. Quality of the image obtained by capturing an object using a digital camera often becomes degraded particularly due to aberration of an imaging optical system.
A blur component is generated in the image due to spherical aberration, comatic aberration, image plane curvature, astigmatism, and the like occurring in the imaging optical system. The blur component of the image due to such types of aberration indicates the one in which a light flux output from a point of the object which in theory converges at a point when there is no aberration and effect of diffraction becomes spread and forms the image. Such a phenomenon optically referred to as a point spread function (PSF) will be hereinafter referred to as the blur component in the image. For example, a defocused image is blurred. However, the blur due to the effect of the aberration in the imaging optical system as described above even when the image is in focus will be referred to as the blur component. Further, a color blur in a color image due to axial chromatic aberration, the chromatic spherical aberration, and chromatic comatic aberration may be described as differences in blurring for each wavelength of light.
There is a method for correcting the degradation of the image due to the blur component using the information on an optical transfer function (OTF) of the imaging optical system. Such a method is referred to as image recovery or image restoration. Hereinafter, the process for correcting the image degradation using the information on the OTF of the imaging optical system will be referred to as an image recovery process.
The image recovery process will be described below. When the degraded image is g (x, y), the original image is f (x, y), the PSF which is a Fourier pair of the OTF is h (x, y), the established equation (1) is as follows:g(x,y)=h(x,y)*f(x,y)  (1)where * is convolution, and a coordinate (x, y) is a coordinate on the image.
When the equation (1) is transformed by Fourier transformation to a display format in a frequency surface, a product for each frequency is obtained as indicated by the following equation (2):G(u,v)=H(u,v)·F(u,v)  (2)where H is obtained by Fourier transformation on the PSF and is thus the OTF. A coordinate (u, v) is a coordinate in a two-dimensional frequency plane, i.e., the frequency.
The original image is obtained from the image of which the quality has been degraded due to imaging by dividing both sides of the equation (2) by H as in the following equation (3):G(u,v)/H(u,v)=F(u,v)  (3).
F (u, v) is returned to a real plane by inverse Fourier transformation to obtain the original image f (x, y) as a recovered image.
When the result of the inverse Fourier transformation on H−1 is R, the original image can be similarly obtained by performing convolution processing on the image in the real plane as in the following equation (4):g(x,y)*R(x,y)=f(x,y)  (4).
R (x, y) described above will be referred to as an image recovery filter. Since an actual image includes a noise component, if the image recovery filter generated using a complete inverse of the OTF as described above is employed, the following occurs. The noise component is amplified along with the image of degraded image quality, so that a desired image is not generally obtained. To solve such a problem, there is a method, for example a method using a Weiner filter, for suppressing a recovery rate in a high frequency side of the image according to an intensity ratio between an image signal and a noise signal. The degradation of the image due to the color blur component may be corrected as follows. For example, if amounts of blur for each color component in the image become uniform by correcting the blur component as described above, the color blur component in the image is corrected.
The OTF changes according to an image capturing state, such as a zoom position state, a diaphragm diameter state, and the like. As a result, it is necessary to change the image recovery filter to be used in the image recovery process accordingly. The color blur and migration caused by the optical system can be corrected by performing the above-described process on each of the colors in the color image.
Japanese Patent Application Laid-Open No. 2010-86138 discusses a technique for correcting magnification chromatic aberration using the above-described OTF. More specifically, a magnification chromatic aberration component is detected from color shift in an edge portion of an image after the image recovery process has been performed. The edge portion of the image is then corrected, so that sharpness of the edge portion of the image is changed for each color component according to a degree of recovery.
On the other hand, there is a method for correcting the color blur in a color image by calculating a color blur area from the color image without using optical information, such as the OTF. In such a method, the blur component is removed from the area in the color image in which the color blur is noticeable, such as a saturated portion, so that the color blur is adaptively corrected. Correction is often performed in such a method by limiting a hue (e.g., a purple color) or the area (e.g., in a vicinity of a saturated pixel) to be corrected for preventing excessive correction due to erroneous determination.
Japanese Patent Application Laid-Open No. 2007-133591 discusses a technique related to the above-described correction method. More specifically, an overexposure map indicating saturated areas of an image is compared with a previously recorded chromatic aberration map calculated from a chromatic aberration model for a correction amount of the chromatic aberration. The purple portions in the pixels in the vicinity of the overexposed area are then regarded as the chromatic aberration, and correction, such as reduction of color saturation, is performed.
However, in the technique discussed in Japanese Patent Application Laid-Open No. 2010-86138, the axial chromatic aberration, the color spherical aberration, the color comatic aberration, and the like which are elements of the chromatic aberration other than the magnification chromatic aberration are not detected. It is thus difficult to accurately perform correction. Further, the chromatic aberration other than the magnification chromatic aberration can be corrected in the image recovery process. However, there may be inadequate correction depending on the degree of recovery, and the color blur may not be sufficiently removed in the saturated area in which correction is difficult. Furthermore, in the technique discussed in Japanese Patent Application Laid-Open No. 2007-133591, the area to be corrected is near the overexposed area, and the color to be corrected is limited. As a result, it is difficult to achieve sufficient correction accuracy.